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Abstract 

The  notion  of  the  central  path  plays  an  important  role  in  the  convergence  analysis 
of  interior-point  methods.  Many  interior-point  algorithms  have  been  developed  based 
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whether  such  algorithms  actually  converge  to  the  center  of  the  solution  set  has  remained 
an  open  question.  In  this  paper,  we  demonstrate  that  under  mild  conditions,  when 
the  iteration  sequence  generated  by  a  primal-dual  interior-point  method  converges,  it 
converges  to  the  center  of  the  solution  set. 
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1  Introduction 


We  consider  linear  programs  in  the  standard  form: 


minimize 

T 

C  X 

subject  to 

II 

(1.1) 

x  >  0, 

where  c,  x  G  R" 

,  b  G  Rm,  A  G  Rmxn(m  < 

n )  and  A  has  full  rank  m. 

The  dual  linear 

program  of  (1.1) 

can  be  expressed  in  the  following  symmetric  form 

minimize 

(fy 

subject  to 

By  =  Be, 

(1.2) 

y>  0, 


where  y  G  R"  is  the  vector  of  dual  slack  variables,  d  =  AT(AAT)~1b ,  B  G  R(n_m)xn  has  full 
rank  and  ABT  =  0  (i.e.,  the  columns  of  BT  form  a  basis  for  the  null  space  of  A).  This  form 
of  the  dual  was  introduced  by  Todd  and  Ye  in  [9]. 

It  is  known  that  the  optimality  conditions  for  (1.1)  and  (1.2)  can  be  written  as  a  2n  x  2 n 
nonlinear  system  with  non-negative  variables;  namely, 

=  0,  (x,y)>  0,  (1.3) 

where  X  =  diag( x),  Y  =  diag(y)  and  e  is  the  n-vector  of  all  ones. 

The  feasibility  set  of  problem  (1.3)  is  defined  as 

F  =  {(z,  y)  ■  x,y  e  R",  Ax  =  b,By  =  Be,  (x,  y)  >  0}. 

A  feasible  pair  ( x,y )  G  T  is  said  to  be  strictly  feasible  if  it  is  positive.  In  this  work  we 
assume  that  strictly  feasible  points  exist. 

We  denote  the  solution  set  of  Problem  (1.3)  by 

<S  =  {(z,y)  :  F(x,y)  =  0,(x,y)  >  0}. 


F(x,y)  = 


1  Ax-b  ' 
By  —  Be 


t 


XVe 
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It  is  known  that  the  existence  of  strictly  feasible  points  implies  that  <5  is  bounded  (see  the 
proof  of  Lemma  2.1  in  this  paper).  In  addition,  if  Sx  and  Sy  are  the  primal  and  dual  solution 
sets,  respectively,  then 

S  =  SX  x  Sy. 

In  this  work,  we  are  only  concerned  with  the  case  when  S  is  not  a  singleton  set,  i.e., 

ri(S)  ^  0 

where  rz(<S)  is  the  relative  interior  of  S.  In  this  case,  the  solution  set  S  has  the  following 
structure  (see  [1]  for  a  proof):  (i)  all  points  in  the  relative  interior  satisfy  strict  comple¬ 
mentarity  (and  all  points  on  the  relative  boundary  do  not);  (ii)  the  zero-nonzero  pattern  of 
points  in  the  relative  interior  is  invariant.  Therefore,  for  any  ( x*,y *)  £  rz(<S),  the  following 
index  sets 


1+  =  {?  :  x*  >  0, 1  <  i  <  n}  and  J+  =  {z  :  y*  >  0, 1  <  z  <  n}. 
are  independent  of  the  choice  of  ( x*,y *).  Moreover,  by  strict  complementarity 

/♦Utf-O-V..,-}  and  4+fV»+  =  it' 


Given  w  £  R"  such  that 

w  >  0  and  eTw  =  n , 

the  zn-center  of  the  solution  set  S  is  defined  as 


(a^,  yw )  =  arg  max  w{x ,  y)  :  (x,  y)  £  5} 

where 

r(x,y)  =  n  n  yr- 

ia  it  id  it 

Equivalently,  in  (1.5)  one  can  replace  ipw(x,y)  by  its  logarithm,  i.e., 

In  tl>w(x,y)  =  ^2  wi ln  Xi  +  Y2  wi^nVi- 
id  It  idlt 


(1.4) 


(1.5) 


(1.6) 
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Obviously,  (xw,yw)  £  ri(S)  and  ipw(xw,yw)  >  0.  From  the  structure  of  both  ifw(x,y)  and  <S, 
we  see  that  xw  and  yw  are  the  re-centers  of  the  primal  and  the  dual  solution  sets,  respectively; 
namely, 


xw  —  arg  max  < 

JJ  xXi  :  X  e  sx 

>  and  yw  =  arg  max  < 

f 

n  V?  ■■ytSy 

..e/+ 

The  existence  and  uniqueness  of  re-center  can  be  established  in  a  standard  manner.  For 
more  details,  see  McLinden  [4]  and  Megiddo  [5].  In  [4]  McLinden  called  xw  and  yw  the 
strong  Pareto  optimal  elements  of  Sx  and  Sy,  respectively  (see  Proposition  5). 

For  re  £  Rn  satisfying  (1.4),  the  re-path  of  Problem  (1.3),  parametrized  by  y,  is  defined 

as 

V  =  {(*»,»"(*.))  :  (x*V), ywM)  e  WWViM )e  =  0},  (1.7) 

In  particular,  the  re-path  corresponding  to  re  =  e  is  called  the  central  path. 

The  existence,  differentiability  and  limit  behavior  of  the  re-paths  were  first  studied  by 
McLinden  [4]  in  a  general  setting,  and  later  by  Megiddo  [5]  for  linear  programming  in  par¬ 
ticular.  The  most  striking  result  in  McLinden  [4]  is  perhaps  Theorem  9.  In  the  case  of  linear 
programming,  it  states  that  the  limit  of  the  re-path  as  y  converges  to  zero  is  the  re-center  of 
S,  namely, 

(*«,yw)  =  Hm(*»,y»).  (1.8) 

H->0 

See  also  Proposition  8.2  in  Megiddo  [5]  and  the  discussion  preceding  it. 

The  primal-dual  interior-point  algorithms  considered  in  this  research  can  be  motivated  in 
several  ways,  e.g.,  path-following  or  potential  reduction,  but  in  essence  they  are  all  variants 
of  Newton’s  method.  The  following  generic  algorithmic  framework  includes  a  majority  of 
existing  primal-dual  interior-point  algorithms. 

Algorithm  1  (Generic  Primal-Dual  Algorithm) 

Given  a  strictly  feasible  pair  (x°,y°)  and  w  £  R"  such  that  w  >  0  and  eTw  =  n.  For 
k  =  0, 1, 2, . . .,  do 

(1)  Choose  ak  £  [0, 1)  and  set  yk  =  ak*  nv- . 
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(2)  Solve  the  following  system  for  (Axk ,  Ayk)  : 

1  Ax 


F\x\yk )  =  -F(xk,yk)  +  pk 

\  A  y 


V w  I 


(3)  Choose  a  step-length  ak  —  min(l,  rkak)  for  rk  £  (0, 1)  and 


Ak 

a  — 


min((A’fc)-1  Axk,  (Yk)~1Ayk) 


(1.9) 


(4)  Form  the  new  iterate 


( xk+\yk+1 )  =  (xk,yk)  +  ak(Axk,  Ayk). 


The  standard  choice  for  w  in  Algorithm  1  is  w  =  e.  Since  the  analysis  is  no  more  complicated 
for  general  w,  we  will  carry  out  our  analysis  for  the  general  case. 

A  straightforward  calculation  gives 


F'{x,y) 


A  0 
0  B 
Y  X 


(1.10) 


There  are  two  fundamental  parameters  in  Algorithm  1,  the  centering  parameter  ak  and 
the  step-length  rk  (or  ak).  The  choice  of  step-length  ak  guarantees  (xk+1  ,yk+1)  >  0.  It  is 
easy  to  verify  that  the  iterates  satisfy  the  following  useful  relationships 

xk+lTyk+1  =  (1  -  ak(  1  -  ak))xkTyk  (1.11) 


and 

fcX  f. 

YkAxk  +  XkAyk  =  -XkYke  +  ak— —w.  (1.12) 

n 

The  first  primal-dual  interior-point  algorithm  for  linear  programming  was  constructed  by 
Kojima,  Mizuzo  and  Yoshise  [3].  It  was  based  on  the  idea  of  following  the  central  path  (i.e., 
w  =  e)  which  was  studied  in  Megiddo  [5]  (see  also  Sonnevend  [7]).  Other  early  primal-dual 
interior-point  algorithms  using  the  central  path  include  Monteiro  and  Adler  [6]  and  Todd 
and  Ye  [9]. 
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The  convergence  properties  of  the  duality  gap  sequence  have  been  studied  extensively 
by  many  authors.  On  the  other  hand,  the  convergence  properties  of  the  iteration  sequence 
were  investigated  only  very  recently.  Tapia,  Zhang  and  Ye  [8]  showed  that  under  suitable 
conditions,  if  ak  is  chosen  to  converge  to  zero  fast  enough,  then  the  iteration  sequence  will 
converge  to  a  solution.  At  this  juncture,  it  is  still  unknown  if  the  iteration  sequence  converges 
when  {crk}  does  not  converge  to  zero;  though  in  practice  convergence  seems  to  be  the  norm 
rather  than  the  exception. 

Another  unanswered  question  is  whether  a  path-following  interior-point  algorithm  can 
be  constructed  with  the  property  that  when  the  iteration  sequence  converges,  it  actually 
converges  to  the  center  of  the  solution  set.  In  the  next  section,  we  establish  that  convergence 
to  the  center  of  the  solution  set  is  indeed  attainable. 


2  Main  Result 


We  begin  by  introducing  our  notation.  Define 


=  I  i»‘>  *  =  1,2, 


(2.1) 


and 

u  i= i 

It  follows  from  (4)  of  Algorithm  1,  (1.11)  and  (1.12)  that 

tf+1  =  <j>k{l  -  afc)  +  wtak4>kak  +  AxkA yk(ak)2 

and 

^k+1  =  j>k{l  ~ak  +  akak). 

Therefore, 

<f>k+1  -  Wi<t>k+1  =  (<f>k  -  Wi4>k)(  1  -  ak)  +  AxkAyk(ak)2. 

This  leads  to 

<f>k+1  —  Wi<f>k+1  (f>k  —  Wi<f>k  1  —  ak 


<j)k+ 1 


+ 


AxkAyk  (ak)2 


<f)k  1  —  ak  +  <jkak  <f>k  1  —  ak  +  crkak 


(2.2) 


(2.3) 
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Let 


7?  =  max 

l<t'<n 

k  Ilk 


(j>k 


-  Wi 


Now,  from  (2.3)  and  the  fact  that  <f>k/<j>k  <  n  we  have 


A+i 


< 


a 


i  k  .  k  k 1  +  && 

i  —  aK  <jKaK  i<i<n 


f  |Ax?A yf 


k  \2 


n(afc) 


(2.4) 


(2.5) 


1  /  1  —  £*fc  +  <7fcafc 

For  simplicity,  in  what  follows  we  will  drop  the  superscript  “to”  from  the  points  on  the 
re-path,  i.e.,  we  will  use  the  convention 

=  (xw(fi),yw(fi)). 

The  following  lemma  states  a  very  intuitive  result;  yet  it  is  central  to  the  establishment 
of  our  main  result.  A  rigorous  proof  for  this  lemma  is  in  order. 

Lemma  2.1  Let  {(xk,yk)}  C  T .  Assume  xkTyk  — >  0  and  rjk  — >  0.  Then  {(rrfc,?/fc)}  con¬ 
verges  to  the  w-center  of  the  solution  set,  i.e., 


lim  (xk,yk)  =  (x'k,y'“). 

k—*o o 


Proof:  The  assumption  xkTyk  — *  0  and  (1.8)  imply  that  (recall  ) 

lim  (x(p),y(?))  =  (x-',y~). 

K — ►  OO 


k  _  xkTyk 


Since 


ll(*\v‘)  -  (*“,»*') II  <  IK*4,!!*)  -  (*(**),  y(*‘))ll  +  ll(*(*‘),  »(**))  -  (*”,y“)ll, 

it  suffices  to  prove 

||(**,y*)-(*(^),y(^))||-0.  (2.6) 

Let 

(**,  yk)  =  1  (I*  +  x(}k),  /  +  »(#*)).  (2.7) 


It  can  be  verified  by  substitution  that 


A  0 
0  B 
Yk  Xk 


(  Xk  -x((j)k)  \  _  I 

0  \ 

1 

1 

,  -  }kw  t 
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^  xk  —  x{(j)k ) 

v  yk  -  y(4>k)  ) 

For  (x,y)  >  0,  define 


/ 


XkYke  -  4>k 


w 


(2.8) 


P(x,y)  =  I  -  (XY-1)1/2At(AXY-1At)-1A{XY~1)1/2, 

Hp(x,  y)  =  (XY)-^P(x,y)(XY)-^2, 

Hd(x,y)  =  (XY)-^(I  -  P^^iXY)-1'2. 

It  is  worth  noting  that  P(x,y)  and  /  —  P(x,y)  are  orthogonal  projection  matrices.  Now  it 
follows  from  (2.8)  (see  Lemma  3.1  in  [8],  for  example)  that 

XkHp(xk,yk)(XkYke-^kw)  > 
YkHd(xk,yk)(XkYke  -  4>kw)  ,  ' 

Therefore,  there  exists  a  constant  f3  >  0  such  that 


*  xk  -  x(<f>k)  y 

K  yk-y(4>k)  ) 

\ 

(2.9) 


ll**-*(#‘)ll  <  (2.10) 

llv*-y(**)ll  <  ^IIS*llll#‘ft(**.y‘)l|ij*.  (2.11) 

Clearly,  (2.6)  will  be  proved  if  we  can  show  that  <f>k Hp(xk ,  yk) ,  (j>kHd(xk,yk)  and  (xk,yk)  are 
uniformly  bounded. 

It  is  known  that  the  level  set 


£(£)  =  TO  {(*,y)  :  xTy  <  £} 

is  bounded  for  all  £  >  0.  To  see  this,  observe  that  given  a  fixed  positive  (x,  y)  €  T,  for  any 
(x,  y)  €  £(£),  we  have  (x  —  x)r(y  —  y)  =  0,  which  leads  to 

(y,x)T(x,y)  =  xTy  +  xTy  <  £  +  xTy. 

This  implies  the  boundedness  of  £(£).  In  particular,  the  solution  set  <5  =  £(0)  is  bounded. 
Using  the  relation  ( xk  —  x(<j>k))T(yk  —  y((j>k))  =  0,  we  have 

y(4>k)Txk  +  x{4>kfyk  =  2  **  V, 
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and  hence 


(*‘)  V  =  j(*‘ V  +  v(ik)Txk  +  *(?)  V  +  x(],k)Ty(}k))  =  xkTyk. 

Thus  ( xk,yk )  G  C(xkTyk)  C  C(x°Ty°).  The  uniform  boundedness  of  ( xk,yk )  now  follows 
from  the  boundedness  of  the  level  set  C(x°Ty°). 

To  show  the  uniform  boundedness  of  4>kHp(xk,yk),  it  is  sufficient  to  demonstrate  that 
4>k !(xkyk)  is  uniformly  bounded  for  all  i  because  P(xk ,yk )  is  an  orthogonal  projection  matrix. 
In  view  of  the  fact  that  (xk,yk)  >  0  and  (x(f>k),  y(4>k))  >  0,  we  have  for  all  i 

<j>k  _  4>k  4  <f>k  4 

XiVi  \(xi  +  xi{4>k)){yi  +  yi(<t>k ))  “  xhi  +  4>kWi  <  min  (Wi) ' 

Similarly,  we  can  show  the  uniform  boundedness  of  <j>kHd(xk,yk).  This  completes  the  proof. 

□ 

Now  we  are  ready  to  state  our  main  result. 

Theorem  2.1  Let  {( xk,yk )}  be  generated  by  Algorithm  1  with  the  parameter  choices  {er^} 
and  {afc}.  Assume 

Al  crk  >  a  and  ak  >  a  for  some  a,  a  €  (0, 1); 

A2  min (XkYke)/(xk)Tyk  >  7 /n  for  some  7  €  (0, 1); 

A3  {(xk,yk)}  converges  to  ( x*,y *). 

Then  {(xk,yk)}  converges  to  the  w-center  of  the  solution  set,  i.e.,  (x*,y*)  =  ( xw,yw ). 

Proof:  From  (1.11),  Assumption  Al  implies  that  the  duality  gap  sequence  converges  to  zero. 
Hence,  ( x*,y *)  is  a  solution.  In  addition,  by  Giiler  and  Ye  [2]  Assumption  A2  implies  that 
(x*,y*)  satisfies  strict  complementarity. 

By  Lemma  2.1,  it  suffices  to  prove  {7*}  converges  to  zero.  First  we  prove  that  {r/k} 
cannot  be  bounded  away  from  zero.  Suppose  this  is  not  true,  i.e.  rjk  >  rj  for  some  7  >  0. 
Then  we  can  write  (2.5)  as 

r]k+1  <  f3kgk 


9 


where 


0k 


1  -  ak 


1  —  ak  +  crkak  i<*<«  ^  xkyk  j  7/(1  —  ak  +  akak ) 

A  contradiction  will  arise  if  0k  <  0  for  some  0  G  (0, 1),  which  leads  to  Q-linear  convergence 
of  {r]k}  to  zero. 

Under  Assumption  Al,  there  exist  constants  0'  G  (0, 1)  and  0”  >  0  such  that 

1  -  ak 


( \AxkAy? 


n(ak) 


k\ 2 


(2.12) 


1  —  ak  T-  crkctk 


and 


Hence,  from  (2.12)  we  have 


n 


(ak)2 


<0'  <  1 


<  0"- 


7/(1  —  ak  +  akak) 

|AxfAyf  | 


0k  <  0'  +  0"  max 


l<i<n 


zfyf  j 


(2.13) 


Now  consider  x*  >  0.  Obviously, 


x^+l  ..  /  Axk' 


1  =  lim  =  lim  1  +  ak- 


,  k  ~  , 11111  l  1  1  , 

fc-»oo  X*  k~* oo  ^  x*  / 

This  implies  Axk/xk  — >  0,  because  {a*}  is  bounded  away  from  zero.  Moreover,  it  is  well- 
known  that  Assumption  A2  implies  that  \Ayk /yk\  is  uniformly  bounded  (see  [12],  for  ex¬ 
ample).  On  the  other  hand,  if  x*  =  0,  then  by  strict  complementarity  y*  >  0.  The  same 
argument,  interchanging  the  roles  of  x  and  y,  gives  that  Ayk/yk  —>  0  and  \Axk/xk\  is  uni¬ 
formly  bounded.  Hence 

Axf  A  y\ 

— nr~  -*•  o,  t  = 

x-yf 


Consequently,  for  k  sufficiently  large 


max/J^!t 

i<«<n  (  xiV% 


< 


1-0 ’ 
20"  ' 


However,  it  follows  from  (2.13)  that  for  k  sufficiently  large 


0k  <  0  = 


1  +  /?' 


<  1. 
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This  contradicts  the  hypothesis  rjk  >  r]  >  0.  Hence  {t]k}  cannot  be  bounded  away  from  zero. 
As  a  result,  there  must  exist  a  subsequence  of  {77*}  that  converges  to  zero.  Consequently, 
from  Lemma  2.1  the  corresponding  subsequence  of  the  iteration  sequence  {( xk ,  converges 
to  the  m-center  of  the  solution  set.  But  since  the  iteration  sequence  itself  is  convergent,  the 
entire  sequence  has  to  converge  to  the  tc- center  of  the  solution  set.  This  completes  the  proof. 
□ 

3  Concluding  Remarks 

Assumptions  A1-A2  can  be  easily  achieved.  For  fixed  ak  —  a  £  (0,1),  Assumption  A2  can 
be  enforced  by  properly  choosing  ak  and  at  the  same  time  ensuring  that  {afc}  is  bounded 
away  from  zero.  A  particular  choice  for  ak  is  given  in  [3],  for  example. 

Let  us  put  Theorem  2.1  in  perspective.  We  have  shown,  under  mild  assumptions,  that 
if  {crk\  is  bounded  away  from  zero,  then  whenever  the  iteration  sequence  generated  by 
Algorithm  1  converges,  it  converges  to  the  center  of  the  solution  set.  Very  recently,  Tapia, 
Zhang  and  Ye  [8]  demonstrated,  under  mild  assumptions,  that  if  {<rk}  converges  to  zero  at 
least  /Mi  nearly,  then  the  iteration  sequence  converges  to  a  point  in  the  relative  interior  of 
the  solution  set.  In  addition,  the  rate  of  the  convergence  of  {erfc}  to  zero  is  reflected  in  the 
rate  of  convergence  of  the  iteration  sequence  (see  also  Zhang  and  Tapia  [11]).  It  is  interesting 
that  at  this  juncture  the  price  one  pays  for  attempting  to  obtain  convergence  to  the  center 
of  the  solution  set  is  the  loss  of  a  guarantee  for  the  convergence  of  the  iteration  sequence  and 
the  loss  of  a  guarantee  for  a  fast  convergence  rate.  We  strongly  believe  that  this  latter  loss 
is  a  reality;  however,  based  on  our  numerical  experience,  we  believe  that  the  former  is  most 
likely  not  a  reality.  Hence  convergence  of  the  iteration  sequence  for  the  case  when  {crk}  is 
bounded  below  has  become  an  important  open  question. 
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